Infrared and near-infrared camera hyperframing

ABSTRACT

Systems and techniques for improving the dynamic range of infrared detection systems and for determining physical properties of a graybody are disclosed. For example, a mechanical superframing technique may comprise positioning a first set of filters in the optical path of an infrared camera at a first time, receiving infrared light from an object through the first set of filters at a detector array, acquiring first subframe image data for the object, positioning a second set of filters in the optical path of the infrared camera at a later time, receiving infrared light from the object through the second set of filters, acquiring second subframe image data for the object, and generating first superframe data based on at least some of the first subframe image data and at least some of the second subframe image data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of and claims the benefit of U.S.patent application Ser. No. 11/388,711, filed on Mar. 23, 2006 (nowissued as U.S. Pat. No. 7,606,484), which is incorporated herein byreference in its entirety.

BACKGROUND

1. Field of Invention

This invention generally relates to infrared (and near-infrared) camerasand, more particularly, to enhanced dynamic-range cameras.

2. Related Art

Infrared cameras may be used to image objects and scenes by detectingradiation in the thermal infrared and/or near-infrared range. Ingeneral, the characteristics of this radiation are dependent (in part)on the temperature of the radiating object.

Infrared cameras use detectors to image objects of interest, as singleimages or video. One type of infrared detection system is a focal planearray (FPA). An FPA system uses an array of infrared detectors such asphotodiodes or bolometers, where the output of each detector in thearray is used as intensity information for an associated image pixel.Different types of detectors are sensitive to different wavelengthranges.

For some objects or scenes, there may be a wide range of in-bandbrightness in the field of view of interest. For example, a rocketlaunch scene may include both a cold rocket hardbody and an extremelyhot exhaust plume. Since a typical IR camera has a brightness dynamicrange of about 12-14 bits, it may be impossible to fully span such ascene with a single exposure value. The brightest or hottest parts ofthe image will often be saturated, while the darkest or coolest part ofthe scene may appear black in the image (since the signal is buried inthe noise floor of the camera).

Additionally, in some circumstances the dynamic range of a scene at aparticular time may not be large, but may change substantially in ashort amount of time (for example, a rocket launch scene at a time justprior to launch versus a time just after launch commences). Although asingle exposure value may be appropriate to image the entire scene at aparticular time, if the temperature profile changes substantially, theexposure may not provide for acquisition of adequate image data at alater time.

For a particular IR camera and expected brightness range, imaging may beoptimized by adjusting the camera to an optimal shutter speed orintegration time. However, it may not be possible to fully encompass ascene's brightness (temperature) variations using a single integrationtime.

SUMMARY

Systems and techniques are provided for improved dynamic range detectionof IR and near-IR from one or more targets. In general, in one aspect,an article may comprise a machine-readable medium embodying informationindicative of instructions that when performed by one or more machinesresult in operations comprising positioning a first filter in an opticalpath of an infrared camera at a first time, receiving infrared lightfrom an object through the first filter at a detector array, andacquiring first subframe image data for the object using a firstintegration time based on the receiving the infrared light from theobject through the first filter at the detector array. The operationsmay further comprise positioning a second filter in the optical path ofthe infrared camera at a second later time, receiving infrared lightfrom the object through the second filter at the detector array, andacquiring second subframe image data for the object based on thereceiving the infrared light from the object through the second filterat the detector array. The operations may further comprise generatingfirst superframe image data based on at least some of the first subframeimage data and at least some of the second subframe image data. Thecamera may be a near-infrared camera.

Acquiring second subframe image data for the object may compriseacquiring the second subframe data using the first integration time.Acquiring second subframe image data for the object comprises acquiringthe second subframe data using a second different integration time. Thefirst filter may be configured to attenuate a first amount of light, andthe second filter may be configured to attenuate a second larger amountof light, and generating first superframe image data based on at leastsome of the first subframe image data and the second subframe image datamay comprise generating a first pixel portion of the first superframeimage data using a first pixel portion of the first subframe data if thefirst pixel portion of the first subframe data does not exceed asaturation threshold amount.

In general, in another aspect, a method may include positioning a firstfilter in an optical path of an infrared camera at a first time,receiving infrared light from an object through the first filter at adetector array, and acquiring first subframe image data for the objectusing a first integration time based on the receiving the infrared lightfrom the object through the first filter at the detector array. Themethod may further comprise positioning a second filter in the opticalpath of the infrared camera at a second later time receiving infraredlight from the object through the second filter at the detector array,and acquiring second subframe image data for the object based on thereceiving the infrared light from the object through the second filterat the detector array. The method may further comprise generating firstsuperframe image data based on at least some of the first subframe imagedata and at least some of the second subframe image data.

In general, in another aspect, an infrared camera system may comprise afilter holder configured to hold a plurality of filters and to positioneach of the filters in turn in an optical path of the infrared camera,and a detection system configured to receive light from a first filterof the plurality of filters positioned in the optical path of theinfrared camera at a first time and to generate first subframeinformation indicative thereof. The detection system may be furtherconfigured to receive light from a second filter of the plurality offilters positioned in the optical path of the infrared camera at asecond later time and to generate second subframe information indicativethereof. The camera system may further comprise a processor configuredto receive the first subframe information and the second subframeinformation and to generate first superframe information based on atleast a part of the first subframe information and the second subframeinformation.

The plurality of filters may include at least one neutral densityfilter. The plurality of filters may include at least one bandpassfilter, which may comprise a spike filter. The system may furthercomprise a motor configured to exert torque on the filter holder. Thefilter holder may be generally wheel-shaped, and wherein the motor isconfigured to exert a torque on the filter holder to cause the filterholder to rotate with a first frequency.

In general, in another aspect, an article may comprise amachine-readable medium embodying information indicative of instructionsthat when performed by one or more machines result in operationscomprising positioning a first filter in the optical path of an infraredcamera, the first filter configured to pass a narrow wavelength bandincluding a first wavelength, acquiring first subframe image dataassociated with the first wavelength at a detector using a firstexposure, acquiring second subframe image data associated with the firstwavelength at the detector using the second exposure, and generatingfirst wavelength image data based on at least some of the first subframeimage data and the second subframe image data. The operations mayfurther comprise positioning a second filter in the optical path of theinfrared camera, the second filter configured to pass a narrowwavelength band including a second different wavelength, acquiring imagedata associated with the second wavelength at the detector, andgenerating second wavelength image data based on the image dataassociated with the second wavelength. The operations may furthercomprise determining one or more physical properties of one or moregraybodies using the first wavelength image data and the secondwavelength image data.

Determining one or more physical properties of the one or moregraybodies using the first wavelength image data and the secondwavelength image data may comprise determining the one or more physicalproperties using calibration data. The one or more physical propertiesmay comprise a temperature.

Acquiring the first subframe image data associated with the firstwavelength at the detector using the first exposure may compriseacquiring the first subframe image data at a first integration time, andacquiring the second subframe image data associated with the firstwavelength at the detector using the second exposure may compriseacquiring the first subframe image data at a second differentintegration time.

Acquiring the first subframe image data associated with the firstwavelength at the detector using the first exposure may compriseacquiring the first subframe image data using a first filter with afirst attenuation, and acquiring the second subframe image dataassociated with the first wavelength at the detector using the secondexposure may comprise acquiring the first subframe image data using asecond filter with a second different attenuation.

In general, in another aspect, a method may comprise positioning a firstfilter in the optical path of an infrared camera, the first filterconfigured to pass a narrow wavelength band including a firstwavelength, and acquiring first subframe image data associated with thefirst wavelength at a detector using a first exposure. The method mayfurther comprise acquiring second subframe image data associated withthe first wavelength at the detector using the second exposure, andgenerating first wavelength image data based on at least some of thefirst subframe image data and the second subframe image data. The methodmay further comprise positioning a second filter in the optical path ofthe infrared camera, the second filter configured to pass a narrowwavelength band including a second different wavelength, acquiring imagedata associated with the second wavelength at the detector, generatingfirst wavelength image data based on the image data associated with thesecond wavelength, and determining one or more physical properties ofone or more graybodies using the first wavelength image data and thesecond wavelength image data.

In general, in another aspect, an infrared camera system may comprise afilter holder configured to hold a first spike filter associated with afirst wavelength and a second spike filter associated with a secondwavelength, the filter holder further configured to position the firstspike filter in the optical path of the infrared camera system for afirst time interval and to position the second spike filter in theoptical path of the infrared camera system for a second time interval.The system may further comprise a detection system configured to receivelight from the first spike filter at a first exposure and to generatefirst subframe information indicative thereof, and further configured toreceive light from the first spike filter at a second exposure and togenerate second subframe information indicative thereof. The detectionsystem may be configured to receive light from the second spike filterat the first exposure and to generate third subframe informationindicative thereof, and further configured to receive light from thesecond spike filter at the second exposure and to generate fourthsubframe information indicative thereof.

The system may further comprise a processor configured to receive thefirst subframe information and the second subframe information and togenerate first superframe information associated with the firstwavelength based on at least a part of the first subframe informationand the second subframe information. The processor may further beconfigured to receive the third subframe information and the fourthsubframe information and to generate second superframe informationassociated with the second wavelength based on at least a part of thethird subframe information and the fourth subframe information, theprocessor further configured to determine one or more physicalproperties of a graybody based on the first superframe information andthe second superframe information.

In general, in another aspect, an article may comprise amachine-readable medium embodying information indicative of instructionsthat when performed by one or more machines result in operationscomprising receiving infrared light from an object at a detector andacquiring first subframe image data of the object at a first exposureusing a first technique to control the exposure. The operations mayfurther comprise acquiring second subframe image data of the object at asecond different exposure using the first technique, acquiring thirdsubframe image data of the object at a third exposure using a differenttechnique to control the exposure, acquiring fourth subframe image dataof the object at a fourth exposure using the different technique tocontrol the exposure, and generating superframe image data of the objectusing at least some of the first subframe image data, the secondsubframe image data, the third subframe image data, and the fourthsubframe image data.

The first technique may be an electronic superframing technique. Theelectronic superframing technique may comprise controlling the exposureby varying the integration time. The second technique may comprise amechanical superframing technique. The mechanical superframing techniquemay comprise controlling the exposure by positioning each of a pluralityof filters in the optical path of the infrared light in turn.

In general, in another aspect, a method may comprise receiving infraredlight from an object at a detector, acquiring first subframe image dataof the object at a first exposure using a first technique to control theexposure, acquiring second subframe image data of the object at a seconddifferent exposure using the first technique, acquiring third subframeimage data of the object at a third exposure using a different techniqueto control the exposure, and acquiring fourth subframe image data of theobject at a fourth exposure using the different technique to control theexposure. The method may further comprise generating superframe imagedata of the object using at least some of the first subframe image data,the second subframe image data, the third subframe image data, and thefourth subframe image data.

These and other features and advantages of the present invention will bemore readily apparent from the detailed description of the exemplaryimplementations set forth below taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows four subframes and an associated superframe that may begenerated using electronic superframing;

FIG. 1B shows a series of subframes and associated superframes that maybe generated using electronic superframing;

FIG. 2 is a method that may be used to generate N pixels for asuperframe using image data for a plurality of subframes;

FIGS. 3A to 3C are schematic illustrations of filter holders that may beused for mechanical superframing, according to some embodiments;

FIG. 4 is a schematic block diagram of a camera system, according tosome embodiments;

FIG. 5 is a method that may be used to generate one or more superframesusing mechanical superframing techniques;

FIG. 6A is a method that may be used to generate one or more superframesusing electronic and mechanical superframing techniques;

FIG. 6B shows 2N subframes and an associated superframe that may begenerated using electronic and mechanical superframing techniques;

FIG. 7 illustrates Planck's blackbody equation at three differenttemperatures; and

FIG. 8 shows a method that may be used to determine physical propertiesof a graybody using superframing techniques.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

Systems and techniques provided herein may allow for enhanced dynamicrange capability in thermal IR and near-IR imaging. For simplification,the term “IR” herein encompasses both IR and near-IR spectral ranges,while the term “near-IR” refers particularly to the near-IR part of thespectrum, ranging in wavelength from about 900 to about 1700 nm.

One technique that may be used to increase the dynamic range of an IRimaging system is electronic superframing Superframing refers to aprocess by which image data for a plurality of images (subframes) iscombined to generate a single image with enhanced dynamic range. Each ofthe subframes has a different exposure, which may be controlled (forexample) by changing the shutter speed and/or integration time of theimaging system.

The superframing technique is particularly useful for infrared camerasystems that are used to image scenes with large difference intemperature, such as the rocket launch described above. With priortechniques, in order to obtain an image of the cold hardbody surface,the sensitivity of the camera would initially have to be set to high toproperly image the thermal features of the cold hardbody surface.However, at an exposure where thermal features of the cold hardbody canbe distinguished, the brightness of the exhaust plume generated atignition rapidly saturates the detection elements of the camera.

One technique that may be used for rocket launch imaging is that, justafter ignition, the sensitivity of the camera may be substantiallyreduced. For example, an integration time of several milliseconds may beused to image the hardbody prior to ignition, then the integration timemay be reduced to about 10 microseconds (for an exemplary indiumantimonide (InSb) camera with a 3 to 5 micron bandpass). Although thisallows the thermal profile of the plume to be imaged, the decreasedsensitivity prevents contemporaneous thermal imaging of the much colderhardbody.

Some existing systems use multiple IR cameras set to various integrationtimes on the scene of interest, in order to obtain image data for awider thermal range. However, this solution is complex and introduceserrors (such as error due to parallax).

By contrast, superframing provides a relatively simple and accuratetechnique to perform IR imaging of scenes with large thermaldifferences. Superframing may be accomplished using electronictechniques, mechanical techniques, or a combination.

FIG. 1A illustrates an existing electronic superframing technique. An IRcamera is configured to acquire a first series of images I₁, I₂, I₃, andI₄, which are referred to as “subframes.” Each of the subframes isacquired at a different exposure value. In the example shown in FIG. 1,each of the subframe images I₁, I₂, I₃, and I₄ are acquired usingdifferent exposures, by varying the integration time τ. A set of foursubframes is acquired in a total time T.

If T is small compared to the time over which the scene changes, adataset for images I₁, I₂, I₃, and I₄ may be reduced to provide data fora “superframe,” shown as image I₁′ of FIG. 1A. For a subframe rate R₁ of100 frames per second, and four subframes per superframe (as shown), R₂(the frame rate with subframe cycling, which is also equal to 1/T) is 25frames per second.

Each of the subframes and the superframe include image data for an arrayof pixels. For example, image I₁′ includes sixteen image pixels P₁ toP₁₆, each of which corresponds to one of sixteen pixels in each of thesubframes I₁, I₂, I₃, and I₄ (not shown). The image information for thesubframe pixels is used to generate image information for each of thesuperframe image pixels P₁ to P₁₆. For example, image information for aparticular superframe image pixel may be derived from image informationfor the corresponding pixel in the one of the subframes that providesthe longest integration time (and therefore, highest sensitivity)without exceeding a saturation threshold.

For example, if the portion of the scene being imaged corresponding topixel P₁ is a relatively cool portion of the object to be imaged, imagedata for pixel P₁ of superframe I₁′ may be generated using thecorresponding image data for the subframe with the longest exposuretime. For other pixels of superframe corresponding image data from oneor more of the other subframes may be used to generate image data forthe superframe pixel.

An exemplary application of electronic superframing is as follows. An IRcamera such as an InSb camera may be configured to cycle through fourintegration times in succession (as in FIG. 1A). Because dark currentvalues for each pixel generally change with integration time, eachintegration time may have an associated non-uniformity correction (NUC).The NUCs are referred to as presets, and for the example describedherein are typically numbered between 0 and 3.

The integration times for each preset may be chosen based on a geometricprogression between the desired endpoint integration times. By selectingthe integration times in this manner, the generated exposures will haveplenty of overlap in the radiance ranges that can be accommodated, whilemaintaining a substantially linear response. The desired endpointintegration times may be selected based on the expected properties ofthe scene to be imaged (e.g., thermal range of the scene to be imaged)and/or system constraints.

For example, for an f/2.5 coldshield and 90% transmitting optics, atypical InSb camera images scene temperatures between −20° C. and 50° C.at about 2000 microseconds integration time (assuming unity emissivityfor simplicity). Each of the pixels of the camera may be thought of as“buckets,” with a capacity of about 1.8×10⁷ electrons. After 2000 μsec,a 50° C. target will fill the bucket to the point where its response isstarting to be non-linear. With an integration time of 30 microseconds,the temperature range that can be imaged is between about 90° C. and300° C.

The interval between the limits of 2000 microseconds and 30 microsecondscorresponds to a factor of 66.7. To select additional integration timesbased on geometric progression, the geometric steps are obtained bydividing the integration times in steps of the cube root of 66.7, whichis about 4. For the current example, the selected integration times forthe four presets are shown in Table 1 below.

TABLE 1 Pre-set Number Integration Time (microseconds) 0 2000 1 500 2125 3 30

To obtain data for a single superframe, data for a series of foursubframes having the above integration times is obtained. A datasettaken with a camera configured to implement superframing techniques thusconsists of four-image cycles of images that can be post-processed downto generate a sequence of superframes.

Post-processing may be accomplished in a number of ways. For example,image data corresponding to a particular pixel of a superframe may bethe image data corresponding to the subframe having the best imagequality for that pixel. For the current example, the best image qualitypixel is the pixel associated with the subframe having the highestintegration time without exceeding a saturation threshold. Thesaturation threshold may be the digital count value for which thedetector is still in a linear regime (typically 80% of full scale, orabout 12,000 counts for a 14-bit camera, which has a digitizer rangefrom 0 to 16383 digital counts).

To generate image data for the superframe, software and/or hardware mayimplement a method 200 such as that shown in FIG. 2. At 205, an nthpixel of N total pixels may be selected. For example, the first (n=1)pixel may be the pixel in the corner of the FPA, which may have acoordinate value defined as (0,0). For the nth pixel, the value for thesubframe having the maximum exposure (here, the 2000 microsecond preset)is compared to a threshold at 210. If the value of the subframe pixeldoes not exceed the threshold, that value may then be processed at 208to generate image data for the nth pixel of the superframe. Imageinformation for the next superframe pixel (e.g., the n=2 pixel) may thenbe determined in a similar manner.

At 215, if the pixel value for the subframe having the maximum exposureexceeds the threshold, the value of the nth pixel for the subframehaving the next lower exposure is compared to the threshold (in thisexample, the 500 microsecond preset). If the value of the subframe pixeldoes not exceed the threshold, the value may then be processed at 208 togenerate image data for the nth pixel of the superframe.

The process continues until the pixel value for the subframecorresponding to the minimum exposure is compared to the threshold at220. If the pixel value does not exceed the threshold, the value may beprocessed at 208 to generate image data for the nth pixel of thesuperframe. If the pixel value for the subframe corresponding to theminimum exposure exceeds the threshold, one or more saturationmitigation techniques may optionally be implemented at 225, or thesaturated pixel value may be used to generate image data for the nthpixel of the superframe.

A number of different saturation mitigation techniques may be used. Forexample, for pixel values that exceed the threshold but are less thanfull scale, the pixel values may be corrected using a saturationalgorithm (e.g., a non-linear correction may be made to the pixel valuebased on pre-characterization of the saturated sensors). In anotherexample, pixel data for one or more neighboring pixels may be used togenerate a value for the nth pixel of the superframe (e.g., usinginterpolating techniques).

As noted above, pixel values for the subframes may be processed at 208to generate image data for the superframe pixels. For example, the pixelvalues may be converted from counts to engineering radiance units,typically Watts/(cm²-sr). The resulting radiometric units may becorrelated with the radiant energy emitted by the target of interest;for example, using the following calibration process.

The camera may be exposed to a calibration target of known radiance(e.g., a temperature-controlled blackbody), and the camera responserecorded (e.g., the number of counts recorded for each of the pixels).The source radiance is then varied, new measurements taken, andcalibration information (e.g., a calibration table) is generated.Because IR cameras are generally designed to produce a digital outputsignal that is very linear in the flux reaching the FPA, a linearexpression may be fit to the calibration table and the resultingcalibration applied to pixel count values.

The radiometric calibration described above results in a transferfunction between the digital counts (the output of the camera) and theapparent radiance of the target (in radiometric units). This isgenerally a simple linear function that may be represented by twocoefficients: the slope and intercept of the linear fit. Thecoefficients change in inverse proportion to integration time, where theshorter the integration time, the higher the radiance of a given pixelcount value.

The above-described electronic superframing techniques may provide anumber of benefits over other existing IR imaging techniques. Imagedynamic range is increased, but without sacrificing noise performance,since each pixel in the resulting superframe is selected from thesubframe with the lowest NEI (Noise-equivalent Irradiance) or NEdT(Noise Equivalent Difference of Temperature) while remaining within thelinear portion of the dynamic range of the ROIC (Readout IntegratedCircuit). NEI may be used for near-IR cameras, rather than NEdT,although other embodiments are possible.

However, electronic superframing may not be optimal in somecircumstances. For example, in IR camera systems incorporating FPAs,externally-generated bias signals are generally provided from off-chip.In order to provide a wide range of integration times, the bias levelsneed to be adjusted. However, because the bias signals generally havesettling times greater than typical sub-frame periods, bias signaladjustment is not practical for a cyclical superframe implementation.Therefore, in practice, the range of integration times that may be usedare those that may be obtained with a single set of bias settings. Forexample, in some systems, the integration time may only be varied by afactor of about 100.

Another challenge with electronic superframing is that the maximum framerate is limited by the longest integration time in the cycle. Forexample, for a system configured to run with three presets correspondingto integration times of 10 msec, 1 msec, and 0.1 msec, the maximum framerate is less than 100 Hz (limited by the 10 msec integration time). Forspatially dynamic scenes, subframe image data obtained with longerintegration times may be blurred relative to shorter integration times.

Accordingly, systems and techniques herein provide for improved imagingusing mechanical superframing instead of (or in addition to) electronicsuperframing. In mechanical superframing, a plurality of opticalelements are positioned in the optical path in turn, so that each of aset of subframes is acquired with a different exposure. For example,each of a plurality of filters (such as neutral density filters,bandwidth filters, spike filters and/or other filters) may be positionedin turn between the lens and detector array of an IR camera.

Mechanical superframing may be particularly well-suited to imaging inthe near-IR (NW) spectrum. NIR imaging is important for a number ofapplications. One important application is the imaging of objectsthrough atmosphere, where visible-light images may be unduly distortedby interaction of optical photons with atmospheric constituents. Somesub-bands of NIR (e.g., NIR with wavelengths of about 1.2 microns)travels through the atmosphere relatively unaffected, and may thusprovide more accurate images.

NIR imaging is also an important technique for understanding thephysical properties of materials as a function of temperature, sincein-band radiance in the NIR region of the electromagnetic spectrum is asteep function of temperature in the range of temperatures oftenencountered in research (e.g., between 100° C. and 1000° C.). Since NWscenes change radiance dramatically with temperature, many applicationsrequire variations of exposure values by factors of 1000 or more, whichmay be difficult to obtain with electronic superframing alone.

Further, adding optical elements to an NIR system is not as difficult asin longer wavelength applications. For example, for an InSb or MCT(mercury cadmium telluride) camera operating in the 3-5 micronwavelength band, a room-temperature neutral density or bandpass filtercan emit as much or more radiance onto the FPA as the scene radianceitself, so that the filter elements may need to be cooled to image thedesired scene. By contrast, the NW radiance of a blackbody at roomtemperature is extremely small, on the order of 10⁻⁷ compared to theradiance of a blackbody at 500K. Therefore, the optical elements in anNW system need not be cooled below ambient temperature. This simplifiestheir use for many imaging scenarios.

Mechanical superframing provides additional benefits. For example,neutral density filters can yield extremely large changes in exposure.Filters with up to ND5 (transmission of 10⁻⁵) attenuation can becommercially obtained at relatively low cost. Additionally, theintegration time used by the camera may be selected to be short, so thatapparent motion within a single exposure is substantially frozen, toincrease system frame rates. This is particularly beneficial in sceneswith relatively high-speed motion, where blurring in the image may occurwith longer integration times.

FIG. 3A shows an embodiment of an apparatus for mechanical superframing.A filter support 300 is generally wheel-shaped, and includes a pluralityof mounting features for holding an associated plurality of filters indifferent positions on filter support 300. The mounting features mayinclude flanges that fix the filters in position on filter support 300.

In the illustrated example, filter support 300 holds six neutral densityfilters 301A to 301F. Neutral density filters are numbered according tothe amount of light attenuated by the filter. An ND-0 filter transmitssubstantially all of the light incident on the filter, while an ND-1filter transmits about 10% of the light incident on the filter, an ND-2filter transmits about 1% of the incident light, and so on. In order tospan a broad range of exposure levels, filters 301A to 301F may be ND-0through ND-5 filters.

Filter support 300 is in communication with a motor (not shown) thatspins filter support 300 rapidly about an axis into the page of FIG. 3Ain the direction shown. As filter support 300 spins, each of the filters301A to 301F are positioned in the optical path of the camera in turn.As a result, the exposure varies cyclically, from a maximum value(corresponding to the lowest filter level) to a minimum value(corresponding to the highest filter level).

Filter support 300 may include one or more cutouts 305 to decrease themoment of inertia about the rotation axis, to reduce gyroscopic effectsthat may occur when a camera including filter support 300 is rotated inpitch or yaw while the wheel is spinning. Filter support 300 may alsoinclude timing features 306 (e.g., notches) that may be used to generatea trigger for the camera to acquire image data for a particular filter301.

The rotation frequency of filter support 300 may be selected to obtainthe desired integration rate and frame rate. As the filter wheel speedis increased, the maximum superframe rate increases; however, themaximum integration time decreases proportionally. The rotation rate maythus be selected to obtain the best results for the particularapplication.

FIGS. 3B and 3C show different implementations of filter supports 300.In FIG. 3B, a spoke configuration is used rather than a wheelconfiguration. Each of the filters 301A to 301E are positioned in theoptical path of the camera in turn. In FIG. 3C, a linear configurationof filters 301A to 301E is used. Rather than positioning the filters inthe optical path cyclically, the filters may be positioned in theoptical path in a forward order (e.g., 301A, 301B, 301C, 301D, and301E), and then the reverse order (301E, 301D, 301C, 301B, 301A).Additionally, the time during which each filter is positioned in theoptical path may differ for the embodiment of FIG. 3C.

FIG. 4 shows a schematic diagram of an infrared camera system 406incorporating a filter holder 400 such as filter holder 300 of FIGS. 3Ato 3C. Filter holder 400 includes a plurality of filters, includingfilter 401A and 401D. System 406 includes a camera unit 480 configuredto receive infrared radiation through a lens 485 and to generate signalsindicative of received radiation at a detector 430 such as a FPAdetector.

Camera unit 480 further includes a motor 420 in communication withfilter holder 400, which may be a calibrated stepper motor. Inoperation, motor 420 applies force (e.g., linear force and/or torque) toholder 400 to cause different filters of holder 400 to be positioned inthe optical path of the infrared radiation, between lens 485 anddetector 430. For example, motor 420 may apply torque to a wheel-shapedembodiment of a filter holder 400 to spin filter holder 400 at apre-determined rotational frequency. Camera unit 480 further includesone or more controllers such as controller 470 to control one or moreelements of system 406.

Camera unit 480 further includes a sensor 475 to sense the position offilter holder 400 and to generate a signal indicative thereof. Forexample, when filter holder 400 includes notches indicating the relativeposition of the filter holder, sensor 475 may detect light passingthrough one of the notches, and generate a signal to trigger detector430 to acquire image data for a particular filter 401.

Infrared radiation incident on camera unit 480 through lens 485 isfiltered by the filter positioned in the optical path of the IRradiation (in FIG. 4, filter 401D). In response to a signal from sensor475, controller 470 generates a signal for detector 430 to acquire imagedata associated with the filter.

System 406 further includes one or more information processing systemsconfigured to store and/or process data acquired by camera unit 480. Asshown in FIG. 4, the information processing systems include a computer490 external to camera unit 480, which is in turn in communication withan external display 495. However, at least some information processingfunctionality may be included in camera unit 480.

In operation, system 406 may obtain NIR images of an object or sceneusing a method such as method 500 illustrated in FIG. 5. At 505, NIRinformation is received from the object or scene. For example, NIR lightradiated by the object or scene is received in system 406 via lens 485.

At 510, received NIR light is filtered with a first filter at t₁. Forexample, when a wheel-shaped filter holder is used, a motor appliestorque to the filter holder to spin it at a high rate. At t₁, a firstfilter positioned in the filter holder is positioned in the optical pathof the NIR received from the object or scene of interest.

At t₁, a plurality of detector elements of a detector array acquireimage data for a first subframe I₁ associated with the first filter, at515. The image data may be acquired using an integration time τ₁ (at515). The integration time is about equal to or less than a time duringwhich the first filter is positioned in the optical path of the receivedNIR radiation (e.g., a time equal to or less than an effective width ofthe first filter divided by the speed of the filter wheel at theposition of the first filter).

Similarly, at 520, received NIR light is filtered at a later time t₂during which a second filter is positioned in the optical path of thereceived NIR radiation, and image data for a second subframe I₂associated with the second filter is acquired (again, with anintegration time τ₁). At 525, NIR is filtered and image data is acquiredfor additional subframes associated with any additional filters. At 530,a superframe may be generated using images I₁ to I_(N); for example,using the techniques described above for electronic superframing. At535, image data associated with the generated superframe may bedisplayed and/or stored.

Method 500 may be used to generate a single image (one superframe) ofthe object or scene of interest, or may be repeated to generate a seriesof images (multiple superframes), to be displayed and/or stored. Whenmultiple superframes are generated, they may be displayed and/or storedas video, so that changes of the object or scene over time may bedisplayed, analyzed, etc. For example, the superframe information may beused to make radiometric measurements, so that in-band radiance oftargets may be measured.

As noted above, mechanical superframing may be accomplished usingcommercially available components. For example, commercial IR ND filtersmay be used. A common commercial ND filter is 1″ in diameter. For someembodiments, filters can be mounted between the sensor and the cameralens without vignetting, since the ray cone coming out of most lenses issmaller than the clear aperture of 1″ filters.

For embodiments in which no attenuation of the IR is needed for somesubframes, an open space may be left in the filter holder, or a filtersuch as an ND 0.03 filter may be used. In some circumstances, it isbetter to use a filter than to leave an open space. For example, inorder to keep the back working distance of the lens unchanged, theoptical thickness of the filters should be substantially equal. In orderto keep the optical thickness substantially constant, filters with glasssubstrates and metallic coatings may be used. The metallic coatingreduces the transmitted light signal, but has a negligible effect on theoptical thickness of the filters, since the coating is a small fractionof the total filter thickness. For the case where filters withthicknesses up to 3 mm are used, the current inventors have determinedthat the back working distance of standard C-mount color video lenses issubstantially unchanged at high f/numbers.

The filter choice and other system configuration parameters may placelimits on operational characteristics such as the integration time. Forexample, in an embodiment in which a filter wheel holds 1″ diameterfilters with a clear aperture of about 0.9 inches (where the aperture isreduced by flanges holding the filters in place), the current inventorsmeasured an 11 degree range of rotation of the wheel between positionswhere the filter is not occluding the FPA. At a 15 Hz rotation rate, theduration of an 11 degree rotation is 2.14 msec, based on a calculatedangular velocity of the wheel at 15 Hz of 5400 degrees/sec. Theintegration time for the above system configuration parameters thuslimit the integration time to about 2.14 msec or less. For longerintegration time operation, the angular speed of the wheel may bedecreased. However, as noted above, increasing the maximum integrationtime reduces the attainable frame rate.

The following describes an embodiment that may be used to implement thesystems and techniques described above. An IR camera system may includea commercial camera head, such as an FLIR Systems Phoenix NIR camerahead with a 320×256 pixel array. The head may be connected to an RTIEunit (Real-time Imaging Electronics). The RTIE converts the raw FPA datainto digital image data in the RS-422 standard format. The image data istransmitted to a National Instruments PCI-1422 frame grabber in the hostcomputer.

For the described embodiment, the metal housing of a standard Phoenixhead was replaced with an aluminum box which supports the filter holderbetween the lens interface and the sensor package. The lens interfacewas a standard C-mount thread cut into the front plate of the housing.An Animatics Smartmotor was used to rotate the filter wheel via atoothed-belt drive. The wheel's rotation was measured by an Omronposition sensor that uses an infrared LED and phototransistor to detectthe presence of small slots in the wheel (one slot per filter in thisembodiment). A LabVIEW virtual instrument (VI) written by one of theinventors miming on the host computer controlled the wheel speed, anddetected the one-per-filter trigger signals generated by the Omronsensor. The VI generated a series of pulses timed off the Omron sensor,and these pulses were conditioned into rising-edge TTL level triggerswith an adjustable phase delay that were fed to the external sync inputof the RTIE connected to the Phoenix head.

The above example is one particular embodiment of the systems andtechniques for mechanical superframing that may be used. Many others arepossible. For example, different types of filters may be used, and thefilter holder may position more than one type of filter in the opticalpath of the received IR radiation.

In addition, in some embodiments, multiple techniques for modifying theincoming light to image and/or analyze IR light from an object or scenemay be used. For example, a single superframe may be generated using aplurality of subframes acquired using a first technique to modify theexposure (such as electronic superframing), and a plurality of subframesacquired using a second technique to modify the exposure (such asmechanical superframing). A superframe using two different exposuremodification techniques may be referred to as a “hyperframe.”

FIG. 6A shows an example of a method 600 incorporating both electronicand mechanical superframing techniques, while FIG. 6B shows an exampleof the acquired subframes and an associated superframe. Again, NIRimaging is used as an example, since self-emission of radiation of thefilters in the NIR regime is negligible. Imaging in other regions of thespectrum (such as thermal IR) may also be performed. However, additionaltechniques may need to be used for other spectral regions; for example,the filters may need to be cooled to obtain a satisfactory signal tonoise ratio.

At 605, NIR information is received at a camera system such as thatshown in FIG. 4, which includes a filter holder to position N filters inthe optical path of incoming IR radiation. At 610, incoming NIR light isfiltered at a time t₁. At 615, subframe data for a first subframe I₁associated with a first filter is acquired using an integration time τ₁.At 620, incoming NIR light is filtered at a later time t₂, and subframedata is acquired for a second subframe I₂ associated with a secondfilter, using integration time τ₁. At 625, the procedure is repeated foreach of the N filters, to acquire image data for N subframes.

At 630, the first filter is again positioned in the optical path ofincoming IR radiation at t₁′ (which may be substantially equal to t₁+T,where T is the rotation period of the filter wheel). At 635, image datafor a subframe image I_((N+1)) may be acquired with an integration timeτ₂ different than τ₁. At 640, incoming NIR is filtered with the secondfilter at a later time t₂′, and subframe data is acquired for a subframeI_((N+2)) associated with the second filter at 645, again withintegration time τ₂. At 650, the procedure is repeated until subframedata for N subframes is acquired with integration time τ₂.

At 655, image information for a first superframe I₁′ is generated usingsubframe data for at least one of the subframes I₁ to I_((N+2)), asdescribed above. At 660, the image information for first superframe I₁′may be displayed and/or stored, and the procedure may be concluded, ormay be repeated to acquire additional image information for additionalsuperframes.

FIG. 6B shows the 2N subframes obtained using both electronic andmechanical superframe techniques. In order to generate image data forsuperframe I₁′ associated with the 2N subframes, the camera system maydetermine the best image data for each image pixel in superframe I₁′from the associated pixel data in the 2N subframes, as described above.

Methods such as method 600 of FIG. 6A may provide for an extendeddynamic range, while also providing additional flexibility. For example,the camera system need not use both electronic and mechanicalsuperframing techniques in all circumstances. In some embodiments, thecamera system may be configured to determine whether mechanical orelectronic superframing alone may be sufficient to generate image datawith the desired dynamic range and resolution. If it is, the system mayselect one technique or the other, and acquire superframes with a higherframe rate than when both techniques are used. However, if the systemdetermines that the image data will not be sufficient using a singlesubframe acquisition technique, the system may implement multipletechniques for acquiring subframe data.

The systems and techniques described herein may be applied in a numberof ways. For example, image data may be obtained and displayed and/oranalyzed to determine the thermal properties of the imaged object orscene. Additionally, the techniques may be used to perform thermographyfor one or more graybodies in a scene being imaged. For example, byincorporating different types of filters in one or more filter holderssuch as those illustrated in FIGS. 3A to 3C, properties such as theradiance or temperature of a graybody may be determined, without knowingthe emissivity of the graybody.

Most physical objects emit infrared radiation as graybodies; that is,most objects have an emissivity of less than one. By contrast, ablackbody has an emissivity of one, and for a temperature T emitsradiation according to Planck's blackbody equation:

${W(\lambda)} = {\frac{2\pi\;{hc}^{2}}{\lambda^{5}}\frac{1}{\left( {e^{\frac{hc}{\lambda\;{kT}}} - 1} \right)}}$

Usually, in order to determine properties of a graybody by detectingemitted IR radiation, its emissivity needs to be known. FIG. 7 showsblackbody curves for a blackbody at 300K, 800K, and 3000K. The curvesfor graybodies are shifted by the emissivity of the body, which issometimes a substantially constant function of wavelength. Thus, whilemeasurement of the radiance of a blackbody at a particular wavelengthyields the blackbody's temperature, it is insufficient to determine thetemperature of a graybody (unless the emissivity is also known).

However, FIG. 8 shows a method 800 that may be used with a camera systemsuch as system 406 of FIG. 4 to determine properties such as the IR fluxand/or graybody temperature without knowing the emissivity. At 805, afilter holder with spike filters corresponding to at least twowavelengths in the expected graybody spectrum is provided. A spikefilter is a type of bandpass filter that passes a narrow wavelengthband. In one example, spike filters for various wavelengths between 0.9microns and 1.7 microns may be provided. The camera system may becalibrated with the spike filters installed in the filter holder byacquiring data from a blackbody at one or more known temperatures, sothat the radiance for a particular detector signal is known prior toobtaining image data for a graybody of interest.

At 810, image data may be acquired for a plurality of subframes. Thismay be accomplished using the systems and techniques described above.For example, in one embodiment, each of the spike filters may bepositioned in turn in the optical path of the IR for a first integrationtime (to obtain first subframe data at each wavelength), then for asecond integration time (to obtain second subframe data at eachwavelength), and so on.

In another embodiment, spike filters may be used in addition to NDfilters, where the spike filters allow the acquisition of image data atparticular wavelengths and the ND filters allow the exposure to bealtered to increase the dynamic range of the image data acquisition. Forexample, a first filter holder may include a plurality of spike filterscorresponding to the wavelengths of interest. A second filter holder mayinclude a plurality of ND filters to obtain data within a particulardynamic range. Image data for a first wavelength may be obtained byplacing the desired spike filter in the optical path of the IR, andpositioning each of the ND filters in the optical path in turn. Theresulting subframes may be combined to generate image information for asuperframe associated with the first wavelength. The procedure may berepeated for each of the spike filters.

At 815, the acquired image data may be analyzed to determine the shapeof the graybody curve. At 820, the desired physical properties may bedetermined based on the shape of the graybody curve. For example, thetemperatures of a number of objects in a scene may be determined,without knowing the emissivity of the objects.

In implementations, the above described techniques and their variationsmay be implemented at least partially as computer software instructions.Such instructions may be stored on one or more machine-readable storagemedia or devices and are executed by, e.g., one or more computerprocessors, or cause the machine, to perform the described functions andoperations.

A number of implementations have been described. Although only a fewimplementations have been disclosed in detail above, other modificationsare possible, and this disclosure is intended to cover all suchmodifications, and most particularly, any modification which might bepredictable to a person having ordinary skill in the art. For example,although some techniques may be particularly beneficial for NIR imaging,they need not be restricted to that portion of the electromagneticspectrum. Additionally, the phrases “image data” and “image information”refer to data/information based on received electromagnetic radiationfrom one or more objects, and need not be used to generate an image(e.g., it may be used for radiance information with or withoutgenerating an actual image).

Also, only those claims which use the word “means” are intended to beinterpreted under 35 USC 112, sixth paragraph. Moreover, no limitationsfrom the specification are intended to be read into any claims, unlessthose limitations are expressly included in the claims. Accordingly,other embodiments are within the scope of the following claims.

1. An article comprising a machine-readable medium embodying informationindicative of instructions that when performed by one or more machinesresult in operations comprising: positioning a first filter in anoptical path of an infrared camera, the first filter configured to passa narrow wavelength band including a first wavelength; positioning athird filter in the optical path of the infrared camera, the thirdfilter having a first attenuation; acquiring first subframe image dataassociated with the first wavelength and the first attenuation at adetector of the infrared camera using a first exposure; acquiring secondsubframe image data associated with the first wavelength at the detectorusing a second exposure; generating first wavelength image data based onat least some of the first subframe image data and the second subframeimage data; positioning a second filter in the optical path of theinfrared camera, the second filter configured to pass a narrowwavelength band including a second different wavelength; positioning afourth filter in the optical path of the infrared camera, the fourthfilter having a second attenuation different than the first attenuation;acquiring image data associated with the second wavelength and thesecond attenuation at the detector; generating second wavelength imagedata based on the image data associated with the second wavelength; anddetermining one or more physical properties of one or more graybodieswithin the first and second wavelength image data by using the firstwavelength image data and the second wavelength image data.
 2. Thearticle of claim 1, further comprising calibrating the infrared camerawith the first and second filters using calibration sources at one ormore temperatures to generate calibration data, and wherein thedetermining one or more physical properties of the one or moregraybodies is performed using the calibration data.
 3. The article ofclaim 1, wherein the one or more physical properties comprises atemperature.
 4. The article of claim 1, wherein the acquiring the firstsubframe image data comprises acquiring the first subframe image data ata first integration time, and wherein the acquiring the second subframeimage data comprises acquiring the second subframe image data at asecond different integration time.
 5. The article of claim 1, whereinthe acquiring image data associated with the second wavelength at thedetector comprises: acquiring third subframe image data associated withthe second wavelength at the detector using the first exposure; andacquiring fourth subframe image data associated with the secondwavelength and the second attenuation at the detector using the secondexposure; wherein the generating second wavelength image data is basedon at least some of the third subframe image data and the fourthsubframe image data.
 6. The article of claim 5, further comprising:positioning the fourth filter in the optical path of the infraredcamera, and wherein the acquiring the second subframe image datacomprises acquiring the second subframe image data using the firstfilter and the fourth filter, and positioning the third filter in theoptical path of the infrared camera, and wherein the acquiring the thirdsubframe image data comprises acquiring the third subframe image datausing the second filter and the third filter.
 7. The article of claim 5,further comprising generating first superframe image data based on atleast some of the third subframe image data and at least some of thefourth subframe image data.
 8. The article of claim 5, furthercomprising generating first superframe image data based on at least someof the first, second, third, and fourth subframe image data.
 9. Thearticle of claim 1, further comprising generating first superframe imagedata based on at least some of the first subframe image data and atleast some of the second subframe image data.
 10. A method comprising:positioning a first filter in an optical path of an infrared camera, thefirst filter configured to pass a narrow wavelength band including afirst wavelength; positioning a third filter in the optical path of theinfrared camera, the third filter having a first attenuation; acquiringfirst subframe image data associated with the first wavelength and thefirst attenuation at a detector of the infrared camera using a firstexposure; acquiring second subframe image data associated with the firstwavelength at the detector using a the second exposure; generating firstwavelength image data based on at least some of the first subframe imagedata and the second subframe image data; positioning a second filter inthe optical path of the infrared camera, the second filter configured topass a narrow wavelength band including a second different wavelength;positioning a fourth filter in the optical path of the infrared camera,the fourth filter having a second attenuation different than the firstattenuation; acquiring image data associated with the second wavelengthand the second attenuation at the detector; generating second wavelengthimage data based on the image data associated with the secondwavelength; and determining one or more physical properties of one ormore graybodies within the first and second wavelength image data byusing the first wavelength image data and the second wavelength imagedata.
 11. The method of claim 10, further comprising calibrating theinfrared camera with the first and second filters using calibrationsources at one or more temperatures to generate calibration data, andwherein the determining one or more physical properties of the one ormore graybodies is performed using the calibration data.
 12. The methodof claim 10, wherein the one or more physical properties comprises atemperature.
 13. The method of claim 10, wherein the acquiring the firstsubframe image data comprises acquiring the first subframe image data ata first integration time, and wherein the acquiring the second subframeimage data comprises acquiring the second subframe image data at asecond different integration time.
 14. The method of claim 10, whereinthe acquiring image data associated with the second wavelength at thedetector comprises: acquiring third subframe image data associated withthe second wavelength at the detector using the first exposure; andacquiring fourth subframe image data associated with the secondwavelength and the second attenuation at the detector using the secondexposure; wherein the generating second wavelength image data is basedon at least some of the third subframe image data and the fourthsubframe image data.
 15. The method of claim 14, further comprising:positioning the fourth filter in the optical path of the infraredcamera, and wherein the acquiring the second subframe image datacomprises acquiring the second subframe image data using the firstfilter and the fourth filter, and positioning the third filter in theoptical path of the infrared camera, and wherein the acquiring the thirdsubframe image data comprises acquiring the third subframe image datausing the second filter and the third filter.
 16. The method of claim14, further comprising generating first superframe image data based onat least some of the third subframe image data and at least some of thefourth subframe image data.
 17. The method of claim 14, furthercomprising generating first superframe image data based on at least someof the first, second, third, and fourth subframe image data.
 18. Themethod of claim 10, further comprising generating first superframe imagedata based on at least some of the first subframe image data and atleast some of the second subframe image data.
 19. An infrared camerasystem comprising: a filter holder configured to hold a first spikefilter associated with a first wavelength and a second spike filterassociated with a second wavelength, the filter holder furtherconfigured to position the first spike filter in an optical path of theinfrared camera system for a first time interval and to position thesecond spike filter in the optical path of the infrared camera systemfor a second time interval; a detection system configured to receivelight through the first spike filter at a first exposure to generatefirst subframe information, the detection system further configured toreceive light through the first spike filter at a second exposure togenerate second subframe information, the detection system configured toreceive light through the second spike filter at the first exposure togenerate third subframe information, the detection system furtherconfigured to receive light through the second spike filter at thesecond exposure to generate fourth subframe information; a second filterholder configured to position a third filter in the optical path of theinfrared camera system to provide the first exposure and to position afourth different filter in the optical path of the infrared camerasystem to provide the second exposure; and a processor configured toreceive the first subframe information and the second subframeinformation and to generate first superframe information associated withthe first wavelength based on at least a part of the first subframeinformation and at least a part of the second subframe information, theprocessor further configured to receive the third subframe informationand the fourth subframe information and to generate second superframeinformation associated with the second wavelength based on at least apart of the third subframe information and at least a part of the fourthsubframe information, the processor further configured to determine oneor more physical properties of a graybody within the first and secondsuperframe information based on the first superframe information and thesecond superframe information.
 20. The system of claim 19, wherein theinfrared camera is a near-infrared (NIR) camera, and wherein theprocessor is configured to determine the one or more physicalproperties, which includes a temperature of the graybody, using storedcalibration data.
 21. The system of claim 19, further comprising amemory configured to store calibration data, and wherein the processoris configured to determine the one or more physical properties of agraybody using the calibration data.
 22. The system of claim 19, whereinthe first exposure is based on a first integration time, and the secondexposure is based on a second integration time of the infrared camerasystem.